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ABSTRACT 



The comprehension processes of adult students solving 



two-step problems of comparison were studied using eye-movement 
experiments based on the assumption that eye fixations are 
synchronous with internal cognitive processes. Twenty university 
students each solved 24 two-step mathematical word problems of three 
sentences each with consistent language (CL) or inconsistent language 
(IL) structures. In CL problems, the unknown variable was the subject 
of the second sentence; in IL, the unknown was the object of the 
relational sentence. Raw eye movement data were transformed into 
consecutive fixations. Overall results were generally in line with 
the model of A. Lewis and R. Mayer (1987) (LK model) suggesting the 
increased difficulty of IL problems as indicated by increased time 
required to process the sentences and the number of errors of 
reversal of correct mathematical operation. Apparently, just as it is 
the case for the performance variable error rate, the effect of 
inconsistent language oh aspects of the solution process (such as 
total solution time, initial reading time, and fixation time on the 
relational clause) is exhibited only when the compare problems have 
to be processed and solved under rather heavy cognitive demands. Some 
theoretical conceptions are developed as alternatives to parts of the 
LM model. One table and one bar graph supplement the discussion. 
(SLD) 
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STUDENTS' C0MFREBEK4I0N FROCSSSBS 
WHEN SOLVING TWO- STEP COMPARE PROBLEMS 

Erik Oe Corte, Lieven Verschaffel, and Ann Paowels 
Center for Instructional Psychology and Technology (CIP&T) 
University of Leaven, Belgium 

Objectives and theoretical background 

At the 1990 AERA Conference, we presentsd two eye -movement experiments 
(De Corte, Verschaffel & Pauwels. 1990), that tested Lewis and Mayer's 
(1987) model of comprehension processes when solving compare problems. This 
theoretical model focusses on two forms of compare problems, namely 
consistent and inconsistent language problems. In a consistent language 
problem (e.g., 'Joe has 5 marbles. Tom has 3 more marbles than Joe. How 
many marbles does Tom have?") the unknown variable is the subject of the 
second sentence, and the relational term (in this case 'more than") is 
consistent with the required arithmetic operation (namely addition). In an 
inconsistent language problem (e.g., "Joe has 8 marbles. He nas 5 marbles 
less than Tom. How many marbles does Tom have?") the unknown variable is 
the object of the second sentence, and the relational term ("less than") is 
in conflict with the arithmetic operation (namely addition). Most empirical 
studies on word problem solving revealed that inconsistent language (IL) 
problems are more difficult than consistent language (CL) problems. To 
explain this finding, Lewis and Mayer (1987) put forward the consistency 
hypothesis : Problem solvers would have a preference for the order in which 
the relational information is presented. In particular, they would prefer 
the order of consistent language problems, in which the unknown variable is 
the subject of the second sentence. When given an inconsistent language 
problem, in which the unknown variable is the object of the relational 
sentence, pupils are assumed to mentally rearrange the relational sentence 
until it fits the preferred format. 

This rearrangement procedure consists of reversing the subject and the 
object of the relational sentence, as well as the arithmetic operation 
suggested by its relational term. Because the comprehension and solution 
process may be more error prone when information must be rearranged, the 
probability of a reversal error (subtraction instead of addition, or the 
reverse) will be greater for IL than for CL problems* 

In an attempt to test their hypothesis, Lewis and Mayer (1987) asked a 
group of adult students to represent and to solve two-step compare 
problems. The results were in line with their prediction. The subjects were 
more likely to miscomprehend the relational statement, and to commit a 
reversal error when the unknown variable was the object of the relational 
sentence than when it was the subject. Moreover, they found that the 
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students made more reversal errors on XL problems when the necessary 
operation was addition or multiplication rather than subtraction or 
division. This interaction was explained by the presence of marked terms 
(e.g., "less") in inconsistent language addition and multiplication 
problems. Since marked terms are more salient than unmarked terms (e.g., 
•more"), the subjects would be more likely to resist reversing them. 

In order to provide a more straightforward test of the hypotheses 
concerning the internal processes involved in this model, we carried out 
two eye-movement experiments in which adult students and third-graders were 
administered one-step addition and subtraction compare problems. The use of 
eye-movement registration in problem- solving research is based on the 
general assumption that internal, cognitive processes are synchronous with 
eye fixations (Just & Carpenter, 1987). In the context of the present 
studies, this means that the fixation time spent on a particular sentence 
of a word problem reflects the time needed to process that clause. 

While the data of the first experiment with adult subjects revealed no 
evidence in favor of the consistency hypothesis, the results of the second 
study with third-graders provided good support of the Lewis and Mayer 
(1987) model. The most plausible explanation for this contradiction is that 
one-step addition and subtraction problems are too easy problems for 
adults. Probably, Lewis and Mayer's model only reflects the solution 
processes of subjects for whom the compare problems that are administered 
have a certain level of difficulty. Therefore, we decidtd to collect eye- 
movements of university students while solving more difficult two-step 
compare problems . 



Hypotheses 

The hypotheses that we derived from the LM-model concern (a) the number 
of reversal errors, (b) response times, and (c) fixation times on the 
different sentences in a problem. 

First of all, the LM-model predicts that problem solvers will make more 
reversal errors on IL than on CL problems. Furthermore, an interaction 
effect between language consistency and the operation to perform was 
predicted: More reversal errors would occur on IL problems involving 
addition or multiplication (- problems containing a marked t^rm) than on IL 
problems involving subtraction or division (- problems containing an 
unmarked term)t this difference would not exist for CL problems. 

With respect to the response times the following hypothesis was stated. 
If students indeed have to reorganize the relational sentence in an IL 
problem to make it similar to their preferred format, this rearrangement 
will necessarily lead to greater response times for IL that for CL 
problems . 
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The LM-model involves that the difference in the solution process 
between CL and IL problems concerns the processing of the second, 
relational sentence i In the case of XX* problems this sentence has to be 
reorganized, in the case of CL problems not. Assuming that there is a close 
link between what is fixated and what is internally processed, we predicted 
that the longer response times for the XL problems would be due to longer 
fixation , ymej on the second, relational sentence and that the fixation 
times on the other two sentences would not differ significantly for both 
kinds of problems. 

According to LM-model the rearrangement subprocedure is already 
initiated immediately after the first reading of the second sentence, i.e., 
before the subject turns to the third sentence in the problem. This implies 
that the above-mentioned difference In the response time and the fixation 
time between CL and IL problems will already have shown up during the 
initial reading of the first and the second sentence . In the present 
article, we call this initial reading of the first and the second 
sentence - which constitutes the core of the LM model - the first phase of 
the word problem solving process. 



Method 
Subjects 

Twenty university students participated in the experiment. 

Task 

Each subject was given 24 two-step compare problems. The first part of 
the problem had a compare structure, the second part was a direct variation 
{e.g., the price of a particular amount of pounds or litres, given the 
price of one pound or litre). Half of the problems had a CL compare 
structure, half of them an IL compare structure. Within both categories 
there were 3 addition and 3 multiplication problems (we will refer to these 
problems as increase problems) and 3 subtraction and 3 division problems 
(we will refer to these problems as decrease problems). So there were eight 
different types of problems. All problems consisted of three sentences, 
which were controlled for the number of characters. Moreover, the same 
cover stories weie used in the different problems types. Because the 
available eye-movement-registration equipment requires that the subject's 
head remains relatively stable during the whole solution process, we bad to 
work with particular number triples, so that the correct answer could be 
easily obtained by mental computation. Examples of the problems are given 
in Table 1. 



ERIC 



6 



4 



Insert Table 1 about here 



Procedure 

All subjects were tested individually. They were asked to give the 
correct answer for each problem, and they were allowed to do this at their 
own pace. The word problems were presented on a television screen placed 2 
meters form the subjects. While they were reading and solving the problems, 
their eye-movements were registered with DEBIC 80, a system that is based 
on the pupil center-corneal reflection method. Every 20 milliseconds the 
spatial coordinates of the subject's point of regard, and a time code are 
stored on an on-line PDP-11 computer. Furthermore, a videorecording of the 
subsequent points of regard on the displayed problems is also availab.e. 
These points are represented as the intersection of a vertical and a 
horizontal axis superimposed on the problem. (For a detailed description on 
the eye- tracking system, see De Graef, Van Rensbergen, and d'Ydewalle, 
1985). 

The raw eye-movement data were transformed into consecutive fixations. 
A fixation was operationally defined as a time period of a minimum of 100 
milliseconds during which the eye is close to immobile at a particular 
location (Rayner, 1978). Starting from those data, the absolute fixation 
times on the three different problem sentences were calculated. These 
variables, as well as the response times, were subjected to an ANOVA with 
language consistency (consistent versus inconsistent) as the independent 
variable. The number of reversal errors was subjected to an ANOVA with 
language consistency and the direction of change (increases or decreases) 
as independent variables (2x2 repeated measures factorial design). 

We point to the fact that response time is not the sum of the fixations 
on the three different sentences of the problem. Indeed, response time 
includes also the saccades (the time intervals between two successive 
fixations), the so-called "missings" (short periods during which there is 
no eye-movement registration, e.g., because the subject blinks eyes or 
looks away from the television screen), and all fixations above, under and 
besides the three areas on the screen which contain a problem sentence. 

To test the hypotheses concerning the first phase of the solution 
process, we tried to decompose each eye-movement protocol in two phases. 
The first phase consists of the initial reading of the first two problem 
sentences; it starts from the presentation of the problem on the screen, 
includes all the fixations on the first two sentences, and ends just before 
the first fixation on the third sentence. The second phase starts from that 
first fixation on the third sentence and ends at the moment the answer is 
given. Eye-movement protocols which did not show a series of (at least two) 
subsequent fixations on the first and on the second sentence before the 
reading of the third sentence began, were excluded from this analysis. 
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Results 

Number of reversal errors 

The results of the analysis on the number of reversal errors were 
completely in line with those obtained by Lewis and Mayer (1987). The total 
uumber of reversal errors for the different types of problems is summarized 
in Figure 1. 



Insert Figure 1 about here 



As predicted, the ANOVA revealed a significant effect of language 
consistency. F(l,19) - 8.39, MS e - 0.29, £ < 0.01, in which IL problems 
produced more reversal errors that CL problems. Furthermore, the analysis 
revealed a significant main operation effect, F(l,19) - 8,64 MS e - 0.14, p_ 
< 0.01: increases produced more reversal errors than decreases. Finally, a 
significant interaction between both independent variables was found, 
F(l,19) - 15.55, MS e - 0.26, p. < 0.01: The difficulty of overcoming 
inconsisent language was enhanced when an increase was also present. 



Total solution process 

Duration . Based on the LM-model , vre predicted that the students would 
need more time to solve the IL problems than to solve the CL ones. The 
results of the ANOVA were in line with this prediction. We found a 
significant main effect of lang lge consistency, F(l,19) » 5.73, MS e » 
32.6, j> < 0.05: IL problems elicited longer response times than CL problems 
(an average of 12.25 s to 11.01 s). 

Fixation time on the different sentences . According to the model, the 
longer response times for IL problems are especially due to the 
reorganization of the relational sentence. Therefore, we predicted longer 
fixation times only on the second, relational sentence of the IL problems. 

The ANOVA of the second sentence indeed revealed a significant effect 
of language consistency, £(1,19) - 5.59, MS, - 3993755.1, £ < 0.05: The 
students spent an average of 2476 msec on the relational sentence in an IL 
problem, and only 2044 msec on that sentence in a CL problem. The fixation 
times on the first sentence (1,865 msec for CL and 2,074 msec for IL, 
£(1,19) » 2.34, MS e - 2,245,041, £ > 0.05) and on the third sentence (1,286 
msec for CL and 1,379 msec for IL, F (1,19) - 2.54, MS e - 413,707, p. > 
0.05) did not differ significantly for both kinds of problem. 
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First phase of the solution process 

The LM-model assumes that the rearrangement of the relational sentence 
in an IL problem takes place before the subject reads the third sentence of 
the problem. In view of testing this hypothesis, we tried to c'acompose the 
eye-movement protocols into two different phases. We decided to include in 
this analysis only those eye-movement protocols which showed several 
subsequent fixations on the first and on the second sentence before the eye 
moved to the third sentence. A significant number of eye-movement protocols 
did not meet this criterion, namely 80 out of 480. In a number of cases the 
eye-movement pattern was so chaotic that there were no successive fixations 
neither on the first nor on the second sentence; in other cases, there were 
several fixations on the third sentence before the first and/or the second 
one was being fixated. The 400 protocols that did meet the creterion were 
equally divided among the CL and the IL problems: 198 for the CL problems 
and 202 for the IL problems. 

Duration of the first phase . In accordance with the prediction, we 
found a main effect of language consistency, F (1,19) - 4.71, MS e - 15.6, £ 

< 0.05. The initial phase lasted significantly longer for IL than for CL 
problems (8.37 s compared to 7.61 s). 

Fixation time on the different sentences during the first phase . In 
line with the LM-model, we expected that during the initial stage of the 
solution process, the effect of language consistency on the absolute 
fixation times would appear only on the second, and not on the first 
sentence. Mean absolute fixation time on the first sentence was 1,800 msec 
for CL and 1,960 msec for IL problems, F (1,19) - 1.48, MS C - 90,592, £ > 
0.05. For the second sentence the means were 1,842 msec and 2,100 msec 
respectively. F (1,19) - 3.43, MS e » 293,291, £ > 0.05. While the 
difference between the means was somewhat larger for the IL than for the CL 
problems, the effect of language consistency on the absolute fixation time 
on both sentences failed to reach significance (for the second sentences £ 

< 0.08). 



Discussion 

Taking into account the results of the present experiment, it seems 
reasonable to conclude that the lack of confirmation of the LM-model 
observed in our earlier study with adults was mainly due to the extremely 
low difficulty level of the problems for the subjects. Indeed, in the 
second experiment with third graders and in the present investigation, in 
which problems were substantially more difficult to the subjects than in 
the first study, the results were very well in line with the LM-model and 
with the performance data collected by Lewis and Mayer (1987) themselves. 
Apparently, just as it is the case for the performance variable "er^r 
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rate", the effect of inconsistent language on such aspects of the solution 
process as 'total solution time", "initial reading time* and "fixation time 
on the relational clause" , only shows up when the compare problems have to 
be processed and solved under rather heavy cognitive demands. 

While the overall results of our eye-movement studies can be 
interpreted as additional empirical evidence for the LM-model, further 
theoretical and empirical work is certainly warranted. 

From a theoretical point of view, there is a strong need to analyse the 
relationship between Lewis and Mayer's conception and operational isat ion of 
the CL/IL distinction for compare problems on the one hand, and several 
more general theoretical notions on the other. In order to initiate a 
critical discussion of the LM-analysis and to provide a perspective for 
further empirical work, we will briefly present these alternative 
theoretical conceptions. 

First, there is Van Dijk and Kintsch f s (1983) idea of text coherence. 
According to this notion, interpretation and integration of new textual 
information about an agent is facilitated when this new information starts 
with the same agent. This idea leads to predictions that are incongruent 
with the LM-model- For example, one could predict that subjects vill have 
less difficulty in reading and understanding the first two sentences in a 
IL problem (where the relational sentence starts with a different agent). 
The results of the present study and of our earlier experiment with third 
graders contradict this alternative prediction, but those of our first 
investigation with adults are in line with it- We remind that in the latter 
study, university students needed not less but significantly more time to 
read the first two sentences of CL problems than of IL problems; this 
outcome is in line with the prediction derived from the theory of Van Dijk 
and Kintsch (1983). 

Related to Van Dijk and Kintsch* s (1983) idea of text coherence is 
Reusser f s (1989) notion of the narrative focus of a word problem, which 
refers to the perspective from which a word problem is stated. Reusser 
(1989) distinguishes between the narrative focus of the story episode 
(i.e., the first part of a word problem containing the givens) ar.d of the 
final question. We restrict ourselves to the first aspect. The narrative 
focus of a story episode can be either protagonist-related or coactor- 
related. It is protagonist-related if the same person or thing in every 
partial action or state is in topic position, i.e.* serves as the 
grammatical subject of all sentences of the story episode (e.g., "Yesterday 
Rudy got 8 marbles from Daniel; today Rudy got some marbles from Hannah; 
now Rudy has 11 marbles"). The narrative focus is called coactor-related if 
different persons or things serve as the grammatical subject of the 
respective sentences of the story episode (e.g., "Yesterday Daniel gave 8 
marbles to Rudy; today Hannah gave some marbles to Rudy; now Rudy has 11 
marbles"). According to Reusser (1989), story episodes with a protagonist- 
related narrative focus are easier than episodes with a coactor-related 
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narrative focus- Applied to our compare problems, this would imply again 
that the first two sentences of an XL problem (which are protagonist- 
related) should be easier to understand than those of a CL problem (which 
are coactor related). The unexpected results of our first experiment with 
adults are in line with this alternative prediction. 

Besides these two theoretical notions involving predictions that are 
incongruent with the LM-model , there are also two other interpretative 
concepts which lead to predictions that are in accordance with it, and can 
therefore be considered as alternative theoretical accounts for the 
positive results obtained in the present study and in the previous 
experiment with third graders. 

First there is the concept of pronominal reference. A closer look at 
the word problems used by Lewis and Mayer (1987) and in our studies, 
reveals that only the IL problems contain pronouns. It is generally 
acknowledged that resolving the problem of pronominal reference may cause 
special difficulties, especially for young, inexperienced readers* So, one 
could argue that the systematic difference in the use of pronouns between 
IL and CL problems may have contributed to the positive results, especially 
in the experiment with third graders* 

Second, a basic assumption underlying the LM-model is that reversal 
errors on IL compare problems are due to difficulties in understanding and 
representing the relational information in the problem statement. However, 
reversa 1 errors on word problems may also originate differently* Take, for 
example, the so-called key-word strategy. Subjects applying this strategy 
do not try to understand and represent the problem statement; they simply 
look for key words, i.e., words with which a particular arithmetic 
operation is associated • For example, the words "altogether' and "more" are 
associated with addition; the words "lost* and "less" with subtraction 
(Nesher & Teubal, 1975). For word problems containing a key word that is 
associated with the correct operation, the key-word strategy yields 
success, but for those having a key word that operates as a distractor, 
this strategy results in reversal errors. In the context of the present 
studies, applying a key-word strategy would lead to correct answers on all 
CL problems, but would produce reversal errors on all IL problems. 
Therefore, one could argue that the differences in success rate between CL 
and IL problems observed in the present investigations and in the study 
with third graders, may have been at least partially due to the application 
of this strategy by some subjects. 

Taking all this into account, it seems necessary to critically 
reconsider the LM-model. However, this theoretical re-analysis should be 
accompanied by further empirical research aiming at elucidating the 
relevance of the alternative concepts mentioned above. In this respect, 
investigators should pay special attention at systematically controlling 
and /or varying all possibly relevant aspects of the statement of compare 
problems, such as the presence of pronouns, the order of presentation of 
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the subject and the object in the relational statement, the place of the 
question sentence, etc. But investigators should also try to complement the 
kind of data used so far (i.e., solution accuracy, solution time, and 
fixation times on different parts of the problem statement) with other 
kinds of empirical findings. In this respect, we think not only of other 
types of eye-movement data {such as the amount of so-called "regressions" 
within and between problem sentences as an indication of comprehension 
difficulty), but also of verbal data such as thinking-aloud protocols, 
retrospective reports, or retelling data (see e.g., De Corte & Verschaffel, 
1987) . 
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Table 1 



Examples of the two-step Compare problems presented in Experiment 3 



Operation 



Problems 



Consistent Language 



Addition 



At Aldi a pound of butter costs 80 Bfr 

At GB butter is 20 Bfr. mor** per pound than at Aldi 

How much do 5 pounds of bv**^r cost at GB? 



Subtraction 



At GB a litre of milk costs 25 Bfr. 

At Aldi milk is 25 Bfr. less per litre than at GB, 

How much do 10 litre of milk cost at Aldi? 



Multiplication 



Nopri sells 150 eggs a day* 

GB sells 2 times as many eggt as Nopri, 

How many eggs does GB sell on 2 days? 



Division 



Nopri sells 50 pounds of tomatoes a day. 

GB sells 1/5 as many tomatoes as Nopri, 

How many pounds of tomatoes does GB sell on 4 days? 



Inconsistent Language 



Addition 



At Nopri a pound oi fi*ars costs 65 Bfr. 

That is 15 Bfr. less per pound than at Coiruyt 

How much do 10 pounds of pears cost at Colreyt? 



Subtraction 



At Coiruyt a prund of sugar costs 40 Bfr. 

That is 10 Bf» . more per pound than at Delhaize 

How much do 5 pounds of sugar cost at Delhaize? 



MuJ tiplication 



Delhaize sells 50 newspapers a day. 

That is 1/5 as many newspapers at GB sells. 

How many newspapers does GB sell on 4 days? 



Division 



Aldi sells 120 litre of water a day. 

That is 2 times as many litres as Nopri sells. 

How many litres of water does Nopri sell on 5 days? 
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Figure 1. The proportion of reversal errors on CL and IL problems for 
each of the 4 operations 
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